We propose and study the problem of distribution-preserving lossy compression. Motivated by recent advances in extreme image compression which allow to maintain artifact-free reconstructions even at very low bitrates, we propose to optimize the rate-distortion tradeoff under the constraint that the reconstructed samples follow the distribution of the training data. The resulting compression system recovers both ends of the spectrum: On one hand, at zero bitrate it learns a generative model of the data, and at high enough bitrates it achieves perfect reconstruction.

The paper proposes a novel problem formulation for lossy compression, namely distribution-preserving lossy compression (DPLC). For a rate constrained lossy compression scheme, for large enough rate of the compression scheme, it is possible to (almost) exactly reconstruct the original signal from its compressed version. However, as the rate gets smaller, the reconstructed signal necessarily has very high distortion. The DPLC formulation aims to alleviate this issue by enforcing an additional constraint during the design of the encoder and the decoder of the compression scheme. The constraint requires that irrespective of the rate of the compression the distribution of the reconstructed signal is the same as that of the original signal.

We propose and study the problem of distribution-preserving lossy compression. Motivated by recent advances in extreme image compression which allow to maintain artifact-free reconstructions even at very low bitrates, we propose to optimize the rate-distortion tradeoff under the constraint that the reconstructed samples follow the distribution of the training data. The resulting compression system recovers both ends of the spectrum: On one hand, at zero bitrate it learns a generative model of the data, and at high enough bitrates it achieves perfect reconstruction. We study several methods to approximately solve the proposed optimization problem, including a novel combination of Wasserstein GAN and Wasserstein Autoencoder, and present an extensive theoretical and empirical characterization of the proposed compression systems. Papers published at the Neural Information Processing Systems Conference.